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REMARKS 

Claims 1-19 are now present in this application. It 
should be noted that the amendments to original claims 1-5 of 
the present application are non-narrowing amendments, made 
solely to place the claims in proper form for U.S. practice 
and not to overcome any prior art or for any other statutory 
considerations . 
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CONCLUSION 



Accordingly, in view of the above amendments and remarks, 
an early indication of the allowability of each of claims 1-19 
in connection with the present application is earnestly 
solicited. 

Should there be any outstanding matters that need to be 
resolved in the present application, the Examiner is 
respectfully requested to contact Donald J. Daley at the 
telephone number of the undersigned below. 

If necessary, the Commissioner is hereby authorized in 
this, concurrent, and future replies, to charge payment or 
credit any overpayment to Deposit Account No. 08-0750 for any 
additional fees required under 37 C.F.R. § 1.16 or under 37 
C.F.R. § 1.17; particularly, extension of time fees. 



Respectfully submitted, 



HARNESS, DICKEY & PIERCE, P.L.C. 




P.O. Box 8910 

Reston, Virginia 20195 

(703) 668-8000 
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SUBSTITUTE SPECIFICATION 

Description 

METHOD for navigating OF NAVIGATION - IN THREE-DIMENSIONAL IMAGE 

DATA 

[0001] This application is the national phase under 35 

U.S.C. § 371 of PCT International Application No. 
PCT/EP2004/053041 which has an International filing date of 
November 22, 2004, which designated the United States of 
America and which claims priority on German Patent Application 
number 103 56 272.9 filed November 28, 2003 and on US 
Provisional Patent Application number 60/525,775 filed November 
28, 2003, the entire contents of which is hereby incorporated 
herein by reference. 

Field 

[0002] The invention generally relates to a method for 
navigating in three-dimensional electronic image data. 

Background 

[0003] Electronic image data in more than two spatial 
dimensions (2D) are widely used for the most varied 
applications. Image data in three spatial dimensions (3D) are, 
for example, for three-dimensional simulations of processes, 
design and construction of spatial objects, and for measuring 
and optically reproducing such objects. 

[0004] A particular application is constituted by methods of 
medical imaging technology where patient bodies are examined in 
three dimensions using radiological imaging methods, for 
example, and the three-dimensional examination data are 
acquired for further processing steps. On the one hand, it is 
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possible in this case in diagnostics to identify examined body 
volumes of particular interest, so-called hot spots. In nuclear 
medicine, image regions of increased intensity that indicate 
the presence of a tumor in the region (increased tissue 
activity) are denoted as hot spots. 

[0005] On the other hand, three-dimensional image data of the 
same body from different imaging methods can be brought 
together in a common display, a process termed fusion, in order 
to obtain a more informative image data record that can say 
more. Data from hot spots can play a particular role during 
fusion, since they permit the image data of precisely these 
body volumes from one imaging method to be viewed in the 
context of the image data of another imaging method. Such a 
fused image data record includes the hot spots as a partial 
image data record that can be particularly characterized. 

[0006] An example of this can be, for example, the fusion of 
image data from positron emission tomography (PET) and from 
computed tomography (CT) . The PET data constitute a diagnostic 
data record that includes information relating to specific 
metabolic functions of the patient body, and is therefore also 
denoted as functional image data or a functional data record. 
PET data essentially image soft parts. By contrast, the CT data 
also image anatomic features, such as bone structure, of the 
patient body, and therefore enable a viewer to have a 
substantially better orientation with the aid of the patient's 
anatomy. Consequently, fusing the functional PET data with the 
CT data substantially facilitates the anatomic assignment of 
hot spots identified with the aid of PET. 

[0007] A particular problem with three-dimensional image data 
of all applications resides in the restricted possibilities for 
optical display. It is customary to use two-dimensional display 
units, computer screens as a rule, that offer only restricted 
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possibilities of visualization in three dimensions. Examples of 
what is known are perspective displays, tomograms through 
planes of the object * to be displayed, or rotating displays of 
the object which is visualized either partially in a 
transparent fashion or in a completely compact fashion. 

[0008] A range of techniques that can be used in the way 
described are available for visualizing three-dimensional 
objects, and these are denoted as volume rendering technic 

(VRT) . It is possible, inter alia, to use a maximum intensity 
projection (MIP) that respectively defines as a two-dimensional 
projection pixel the brightest pixel along each line of sight 
going through the three-dimensional object in a fashion 
starting from the (virtual) viewer. Alternatively, it is 
possible to undertake multiplanar reformatting (MPR) , in the 
case of which different two-dimensional projections of the 
object are displayed, for example mutually perpendicular 
projections . 

[0009] The restricted possibilities of optical illustration for 
three-dimensional image data firstly complicate the orientation 
in the objects displayed, since the viewer does not have 
immediate access to the depth information nor, in association 
therewith, to navigation within the data. This problem likewise 
arises in viewing, for example in diagnostic evaluation, as in 
the case of production, for example in three-dimensional 
construction. 

[0010] Methods that use a rotating MIP of a functional data 
record for navigation exist in medical diagnostics. The 
disadvantage in this is that the anatomical assignment is 
thereby not yet always unique, for example when two hot spots 
lie very tightly next to one another. Consequently, these 
methods require a procedure in two stages, which is therefore 
troublesome: firstly, a slice plane through the hot spot of 
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interest is laid on the rotating MIP (one-dimensional 
information) , and then this slice must additionally be 
displayed and the position of the hot spot therein must be 
determined. Not until then is the three-dimensional information 
relating to the position available. 

[0011] Tools for exploring volume data records that comprise 
two-dimensional and three-dimensional input units are known 
from the dissertation entitled "3D-EXPIiORATION VON 
VOLUME NDATEN ; Werkzeuge zur interaktiven Erkundung 

medizinischer Bilddaten" ["THREE-DIMENSIONAL EXPLORATION OF 
VOLUME DATA; Tools for interactive exploration of medical image 
data"] , October 21, 1998, by M. Jahnke. For example, a three- 
dimensional cursor object is proposed by mcano — — which it is 
possible to select a partial volume of a volume data record; 
this partial volume can be understood as a region of interest 
(ROD or voxel of interest (VOI) . The partial volume selected 
in such a way can then be further used as an independent 
viewing volume within which the exploration is continued. 

[0012] A further three-dimensional object proposed there — i-sis 
referred to as the_so-called prober, which constitutes a three- 
dimensional geometric object, for example a cube. The prober 
can be positioned like a cursor. It serves the purpose of 
determining scanned values of the volume respectively 
surrounded by the prober; in the case of a cube, these scanned 
values can be two-dimensional projections of the volume onto 
the cube faces. The tools proposed in the work of M. Jahnke in 
each case serve the manual exploration of partial volumes. 

[0013] It is known from section 2.3.5 of the master's thesis 
entitled "A System for Surgical Planning and Guidance using 
Image Fusion and Interventional MR" by David T. Gering, 
submitted to the Massachusetts Institute of Technology in 
December 1999^ to click on a point of a first two-dimensional 
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projection from a three-dimensional electronic data record in 
order respectively to set the center of the first projection 
and the centers of a second and a third projection having the 
same orientation as the first projection on the clicked points, 
the three projections respectively having different 
magnification factors, as is illustrated in ' figures 2 to 7 
there . 

SUMMARY 

[0014] It is the object of at least one embodiment of the 
invention to specify a method for navigating in three- 
dimensional image data records that automates the finding and 
determination of the three-dimensional position of three- 
dimensional partial image data records of particular interest, 
as well as the visualization thereof, and thereby facilitates 
the same . 

[0012] — Tke — invention achieves — this — object by means — e£ — a method 
having the features of the first patent claim. 

[0015] The features of patent claim l At least one embodiment of 
the invention may yield the advantage that a user can use a 
conventional two-dimensional projection to select a partial 
image data record of particular interest, and thereby 
automatically obtains a further projection of the image data 
record that likewise again includes the partial image data 
record. The user need not firstly manually produce the further 
projection in which he lays a slice plane in the original 
projection. In this sense, the projection of the partial image 
data record is used, as it were, as an active link that can, 
for example, be selected by the user with the aid of a mouse or 
some other pointer unit, that is to say can be clicked on. The 
production of the tomograms required for identifying and for 
determining the position of the partial image data record is 
thereby intuitively configured and simplified. 
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[0016] In an advantageous refinement of at least one embodiment 
of the invention, use is made of an image data record that has 
been formed by fusing at least two source image data records. 
It is thereby possible to facilitate navigation for a user, 
particularly in image data records that have an information 
content extended by the fusion. The extended information 
content, in turn, can be used precisely for the purpose of 
automatically identifying partial image data records of 
possible interest from which the user can then make a manual 
selection . 

[0017] In a further advantageous refinement of at least one 
embodiment of the invention, use is made of partial image data 
records which have all been formed from the same source image 
data record. It is thereby possible to use source image data 
records that are particularly suitable for identifying partial 
image data records possibly of interest, and a user 
automatically knows that the partial image data records have 
been selected with reference to the particular aspects of the 
source image data record used for identification. 

[0018] In a further advantageous refinement of at least one 
embodiment of the invention, use is made as source image data 
records of at least one image data record from a computed 
tomography method and one from a positron emission tomography 
method. This combination is of particular interest with 
reference to medical diagnostics in cancer therapy, since CT 
image data records provide a viewer with the possibility of a 
particularly good orientation within the anatomy of a patient, 
while PET image data records are particularly well suited to 
identifying body volumes possibly at risk of cancer. 

[0019] The method can be executed on a computer. For this 
purpose, it can either be installed on the computer, or it can 
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be designed as a computer program product that facilitates 
executing or installing the method on a computer. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0020] Further advantageous refinements — examp 1 e embodiment s 
of the invention follow from the dependent — patent — claimo — aftd 
the detailed description ei -and the figures. 

[0021] Exemplary — Example embodiments of the invention are 
explained below with the aid of figures, in which: 

figure 1 shows a screenshot with two-dimensional projections 
of a three-dimensional image data record, 

figure 2 shows a screenshot with slice plane and hot spots, 

figure 3 shows a schematic view of a slice plane with hot 
spots , 

figure 4 shows a screenshot with slice plane and hot spots, 

figure 5 shows a schematic view of a slice plane with hot 
spots , 

figure 6 shows a schematic screenshot with random tomograms, 

figure 7 shows a schematic screenshot with tomograms through a 
hot spot, 

figure 8 shows a schematic view of a random slice plane, and 

figure 9 shows a schematic view of a slice plane through a hot 
spot . 

DETAILED DESCRIPTION OF THE EXAMPLE EMBODIMENTS 
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[0022} Figure 1 illustrates a screenshot of a medical image 
processing workstation having two-dimensional projections of a 
three-dimensional image data record. A CT image data record is 
shown that was obtained by fusion with a PET image data record. 
The image data records have been previously registered, that is 
to say have been oriented with one another correctly in terms 
of scale and position, and have been acquired with all items of 
three-dimensional information . 

[0023] The CT data record stands, for example, for a volume 
data record that includes anatomical information, and the PET 
data record stands, for example, for a volume data record of 
the same patient having functional information. The projection 
illustrated was obtained with the aid of the VRT and MIP 
rendering methods, which stand, for example, for any type of 
three-dimensional volume rendering, and with the aid of the MPR 
method, which stands, for example, for any type of slice plane 
rendering . 

[0024] The methods support two forms of so-called clipping 
objects: clip planes (slice planes) and slabs (slice planes of 
defined thickness) . In accordance therewith, the images of the 
functional data record can be both tomograms (being imaged onto 
the slice plane of the clip plane or of the slab) , and volume 
images that are then projected into the clip plane or the slab 
exactly as the anatomical data record. A user can switch back 
and forth between volume display (MIP) and tomogram display 
(MPR) . 

[0025] An important aspect is also the possible parameter 
setting of the rendering methods used. In the case of VRT, 
parameters such as color, transparency or tissue assignment can 
be changed at any time in the CT parameter, in order to achieve 
the optimum view of the CT data record. In the case of MPR or 
MIP, it is possible at any time to change parameters such as 
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windowing, color LUT, masking (that is to say threshold values 
that determine the visibility of information components) . 

[0026] M asking, in particular, is very important in order to 
restrict the display as far as possible only to the hot spots, 
but to mask out further information components of the data 
record which includes the hot spots, in order not to cover up 
too much of the anatomical information of the CT data record. 
Moreover, it is also possible to set a blending factor that 
describes the mixing ratio of CT and PET display. No further 
consideration will be given below to the various parameter 
settings . 

[0027] The display shows a screenshot 1 having four so-called 
viewports 3, 4, 5, 6, that is to say screen segments that can 
respectively show different slice planes or slice layers. Using 
these different projections, which must be manually aligned and 
produced by a user with the aid of the tools and buttons in the 
screen segment 7, a user can display interesting partial image 
data records or hot spots and determine their three-dimensional 
position. For the purpose of better orientation, viewport 6 
offers a rotation view 8, to be operated manually, of the PET 
data record. 

[0028] Figure 2 illustrates a screenshot 1 with slice plane and 
hot spots in a data record which is again fused from CT and PET 
data. The screen 1 shows only a single viewport 11, in which a 
tomogram through the fused image data record is indicated by 
dashes. The projection in the viewport 11 comprioco includes a 
hot spot that can be detected as an optically emphasized image 
section in the abdominal space of the imaged patient body. The 
emphasized image section can be selected by a user, for example 
by mouse click, in order to produce further screenshots. There 
is no need for further manual actions by a user , and so the 
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screen segment 7 includes only a reduced number of buttons and 
tools. 

[0029] Figure 3 is a schematic of the projection from the 
preceding figure. It shows the slice plane 13 through the fused 
image data record that consists of a projection of the CT data 
record 14 and the PET data record 15. Hot spots 19 within the 
PET data record 15 are emphasized optically in a detectable way 
for a user, for example by moans way of particularly bright or 
striking coloring. The slice plane 13 illustrated is positioned 
and orientated such that a user can see the hot spots 19. The 
visualization of the hot spots 19 is f unctionalized in such a 
way that the user can select one of them manually, for example 
by clicking with a mouse. The hot spots 19 lie relatively 
tightly next to one another and are therefore not capable of 
being analyzed by an automated method. 

[0030] Figures 4 and 5 depict illustrations in a fashion 
similar to the preceding figures 2 and 3 with the use of the 
same reference numerals. Instead of a slice plane (clip plane), 
however, a slab 16 is illustrated, being detectable in the 
illustration in figure 5 as a box. Otherwise, reference is made 
to the description of the preceding figures. 

[0031] Figure 6 illustrates a screen view 1 with four viewports 
3, 4, 5, 11. In accordance with the previous description, 
viewport 11 shows a f unctionalized projection to the extent 
that a user can select the hot spots 19 in the slice plane 13, 
for example by a mouse click. Viewports 3, 4, 5 show tomograms 
14, 15 of the fused data record in randomly selected slice 
planes that do not include the hot spots 19. These are included 
only in the slice plane 13 in the tomogram 17. An exact 
localization of the hot spots 19 requires the selection of a 
display that shows the hot spots 19 also in the further, 
different projections in the viewports 3, 4, 5. 
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[0032] To this end, the user firstly selects one of the above- 
described forms of display in order to obtain an optimum view 
of the hot spots 19. At the viewport 11, it is possible to 
rotate spatially into the correct view, and to displace the 
slice plane 13 in order to focus into the hot spot 19. Three- 
dimensional depth information is provided indirectly on this 
navigation image, and enables the hot spot 19 to be selected 
with a mouse click. So-called volume picking, which 
automatically focuses the screen view 1 onto the hot spot 19 
for the user, is initiated by a mouse click on the hot spot 19. 

[0033] Firstly, the clip plane 13 or, if appropriate, the 
centroid of the slab in the viewport 11 is displaced in the 
fusion display into the selected hot spot 19, while secondly 
all other images displayed in the other viewports 3, 4, 5 on 
the screen are likewise displaced into the hot spot 19. 
Consequently, a screen view 1 that is optimum for identifying a 
hot spot 19 is automatically produced for the user without the 
need for him to manually set projections suitable in terms of 
rotation angle and depth in all viewports 3, 4, 5 . As a result 
of the volume picking, the images are displaced from the 
viewports 3, 4, 5 into the selected hot spot 19, as is likewise 
the slice plane 13 in the viewport 11. 

[0034] Consequently, the hot spot 19 can be sought very easily 
in the viewport 11, and it is then possible to focus all four 
viewports 3, 4, 5, 6 onto the hot spot 19 with a mouse click. 

[0035] Using the same reference numerals as in the preceding 
figure, Figure 7 illustrates the screen view 1 obtained as a 
result of volume picking and optimized with regard to the hot 
spot 19 selected by the user. The slice plane 13 is positioned 
in the viewport 11 such that the tomogram 17 shows a projection 
together with hot spot 19. The remaining images now show other 
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mutually referencing tomograms either of a single or the fused 
data records, which likewise respectively include the hot spot 
19. The tomograms reference one another mutually, as is 
indicated by marking lines A, B, C, which run through all 
viewports 3, 4, 5, in each case showing the selected hot spot 
19. The hot spot 19 is thereby displayed in a localized fashion 
that is optimally visible to the user. 

[0036] The process of focusing that was explained in the 
preceding figures 6 and 7 with reference to the viewport 11 
there, is illustrated once more in figures 8 and 9. Figure 8 
illustrates the tomogram of the fused image data record 14, 15 
that includes a hot spot 19. The slice plane 13 is randomly 
positioned and includes no projection of the hot spot 19. In 
figure 9, the slice plane 13 is displaced such that it cuts the 
hot spot 19 and includes a projection 17 of the fused image 
data record. 

[0037] Any of the aforementioned methods may be embodied in the 
form of a system or device, including, but not limited to, any 
of the structure for performing the methodology illustrated in 
the drawings . 

[0038] Further, any of the aforementioned methods may be 
embodied in the form of a program. The program may be stored 
on a computer readable media and is adapted to perform any one 
of the aforementioned methods when run on a computer device (a 
device including a processor) . Thus, the storage medium or 
computer readable medium, is adapted to store information and 
is adapted to interact with a data processing facility or 
computer device to perform the method of any of the above 
mentioned embodiments . 
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[0039] The storage medium may be a built-in medium installed 
inside a computer device main body or a removable medium 
arranged so that it can be separated from the computer device 
main body. Examples of the built-in medium include, but are not 
limited to, rewriteable non-volatile memories, such as ROMs and 
flash memories, and hard disks. Examples of the removable 
medium include, but are not limited to, optical storage media 
such as CD-ROMs and DVDs; magneto-optical storage media, such 
as MPs; magnetism storage media, such as floppy disks 

(trademark), cassette tapes, and removable hard disks; media 
with a built-in rewriteable non-volatile memory, such as memory 
cards; and media with a built-in ROM, such as ROM cassettes. 

[0040] Example embodiments being thus described, it will be 

obvious that the same may be varied in many ways. Such 

variations are not to be regarded as a departure from the 
spirit and scope of the present invention, and all such 
modifications as would be obvious to one skilled in the art are 
intended to be included within the scope of the following 
claims . 
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